1. Use the DISCUSSION tab above to find your discussion questions and/or tasks.
2. POST YOUR GROUP CONSENSUS HERE (The group leader for these 4 Binary Choice activities will be the member whose first name comes earliest in alphabetical order.):
Binary Choice Item # 1 - flaw(s)?The word “they” is the flaw.
Evidence: Commandment 2:"They" can refer to the district administrators or the teachers. This could be supported by Commandment 2. This wording may become confusing to the students.
Binary Choice Item # 2 - flaw(s)?The flaws in #2 are the complicated syntax and the extremely difficult vocabulary.
Evidence: Commandment 4 supports the idea using complex syntax on assessments; as a teacher, you should avoid complicated sentence construction – use very simple sentences. Also support comes from commandment 5. Thou shall not use vocabulary that is more advanced than your audience. As a teacher you might understand what you are asking in the assessment, but you have to remember, will your students understand?
Binary Choice Item # 3 - flaw(s)?The flaw is in the phrase “it is almost impossible”.
Evidence:Commandment 3 clearly states this, and it is relatable because words like 'never' and 'always' add significant meaning to a sentence. These words imply that the sentence is either always true or never true, and this can result in some uncertainty. A good assessment question should concentrate on what the student knows rather then just educated guesses and uncertain answers.
Binary Choice Item # 4 - flaw(s)?The flaws would be the complex syntax and the consistent use of negatives.
Evidence:The sentence is just too wordy and the meaning gets lost in the complex syntax which comes from command 4. The complex syntax has to do with the use of "negatives." By negatives I mean, the words avoid, fail, and never not necessarily "not." Fail is even used twice in the sentence. It makes sense after reading it a few times but you have to simplify the sentence and maybe change those three words in order understand it.
3. POST YOUR GROUP PRODUCT HERE - Multiple Binary-Choice assessment item (the group member whose birthday is closest to Christmas will be the leader for this Multiple Binary-Choice activity : Johnny picked 18 apples, his little sister picked 7, and his parents picked 23 apples together.
Answer True or False 1. Johnny picked 12 more apples than his sister. 2. Together Johnny and his sister picked 25 apples. 3. The parents picked 5 more apples than Johnny. 4. The family picked 47 apples total. 5. If Johnny and his sister ate 3 apples, the family would have 45 apples left.
Answers: 1)false 2)true 3)true 4)false 5)true 4. POST YOUR GROUP CONSENSUS HERE (The group leader for these 4 Multiple Choice activities will be the member whose last name comes earliest in alphabetical order.):
Multiple-Choice Item # 1 - flaw(s)? Choice A is lengthy so the student might think the correct answer would be A. (It's an unintended clue to the right answer-slide #40).
Evidence: This question supports Commandment 3. Thou shall not provide students with unintended clues regarding appropriate responses. Inexperienced test writers often make the correct answer to a multiple-choice item twice as long as the incorrect responses. Students who do not know the answer may infer that if the teacher went into great detail for one answer- there must be something ‘special’ about that choice option.
Multiple-Choice Item # 2 - flaw(s)?This question has an unintended clue occur at the very end. The word "an" gives a clue that the answer must be A.
Evidence:It is clear that this question supports Commandment 3. Thou shall not provide students with unintended clues regarding appropriate responses. The students who know there grammar very well will immediately choose “A” – they do not have to know any information about No Child Left Behind to guess correctly!
Multiple-Choice Item # 3 - flaw(s)?The flaw for this question would be using "All of the above".
Evidence: On slide 43 it states when you include “all-of-the above” – a student who knows only two of the alternatives are correct can guess that the other two are correct as well. In addition, choices A and B are very similar, so it may throw off the students or it may show them that A and B are both correct, so option D is the correct choice.
Multiple-Choice Item # 4 - flaw(s)?This would have to go along with trying to keep all the answers around the same length. D. is significantly longer than the other three. Evidence:Slide 40: Students can figure that if the teacher has given so much attention to constructing an answer options there must be something special about it. Also, this question supports Commandment 3. Thou shall not provide students with unintended clues regarding appropriate responses. Choice D is considerably longer then all the other choices. Students who do not know the answer may infer that if the teacher went into great detail for one answer- there must be something ‘special’ about that choice option.
5. POST YOUR GROUP CONSENSUS HERE (The group leader for this Matching Item activity will be the member whose birthday is closest to Valentine's Day.):
“Find the flaws . . . use evidence from the ‘5 Commandments and/or the Item Writing Guidelines to support your answers.” On the line to the left of each measurement concept listed in Column A, please write the letter of the most relevant measurement procedure in Column B.
Column A
Column B
1. Bias-Detection
A. Determining DIF indices
2. Item Difficulty
B. Computing K-R formula
3. Reliability
C. Evaluating curricular alignment
4. Score Interpretation
D. Deriving percentiles
5. Validity
E. Calculating p// - values
Matching Item # 1 - flaw(s)?First I would mention that there are no extra responses. It is the same on both sides. Next would be the confusing directions. Evidence:On slide 53 it states the directions for an assessment should always make explicit the basis on which matches are to be made – AND, the number of times a response can be used. Slide 51 states...a student who knows 80% of the matches to the premises may be able to figure out the remaining matches - guessing and/or process of elimination. Either way – this does not give you accurate data about the student. Also, Commandment 1 is broken here because of the unclear directions. Commandment 1. Thou shall not provide opaque directions to students. Unclear test-taking directions can result in confused test takers. Most teachers design test items for topics they know very well - and often provide only sketchy directions for students.
1. Use the DISCUSSION tab above to find your discussion questions and/or tasks.
2. POST YOUR GROUP CONSENSUS HERE (The group leader for these 4 Binary Choice activities will be the member whose first name comes earliest in alphabetical order.):
Binary Choice Item # 1 - flaw(s)? The word “they” is the flaw.
Evidence: Commandment 2: "They" can refer to the district administrators or the teachers. This could be supported by Commandment 2. This wording may become confusing to the students.
Binary Choice Item # 2 - flaw(s)? The flaws in #2 are the complicated syntax and the extremely difficult vocabulary.
Evidence: Commandment 4 supports the idea using complex syntax on assessments; as a teacher, you should avoid complicated sentence construction – use very simple sentences. Also support comes from commandment 5. Thou shall not use vocabulary that is more advanced than your audience. As a teacher you might understand what you are asking in the assessment, but you have to remember, will your students understand?
Binary Choice Item # 3 - flaw(s)? The flaw is in the phrase “it is almost impossible”.
Evidence: Commandment 3 clearly states this, and it is relatable because words like 'never' and 'always' add significant meaning to a sentence. These words imply that the sentence is either always true or never true, and this can result in some uncertainty. A good assessment question should concentrate on what the student knows rather then just educated guesses and uncertain answers.
Binary Choice Item # 4 - flaw(s)? The flaws would be the complex syntax and the consistent use of negatives.
Evidence: The sentence is just too wordy and the meaning gets lost in the complex syntax which comes from command 4. The complex syntax has to do with the use of "negatives." By negatives I mean, the words avoid, fail, and never not necessarily "not." Fail is even used twice in the sentence. It makes sense after reading it a few times but you have to simplify the sentence and maybe change those three words in order understand it.
3. POST YOUR GROUP PRODUCT HERE - Multiple Binary-Choice assessment item (the group member whose birthday is closest to Christmas will be the leader for this Multiple Binary-Choice activity :
Johnny picked 18 apples, his little sister picked 7, and his parents picked 23 apples together.
Answer True or False
1. Johnny picked 12 more apples than his sister.
2. Together Johnny and his sister picked 25 apples.
3. The parents picked 5 more apples than Johnny.
4. The family picked 47 apples total.
5. If Johnny and his sister ate 3 apples, the family would have 45 apples left.
Answers: 1)false 2)true 3)true 4)false 5)true
4. POST YOUR GROUP CONSENSUS HERE (The group leader for these 4 Multiple Choice activities will be the member whose last name comes earliest in alphabetical order.):
Multiple-Choice Item # 1 - flaw(s)? Choice A is lengthy so the student might think the correct answer would be A. (It's an unintended clue to the right answer-slide #40).
Evidence: This question supports Commandment 3. Thou shall not provide students with unintended clues regarding appropriate responses. Inexperienced test writers often make the correct answer to a multiple-choice item twice as long as the incorrect responses. Students who do not know the answer may infer that if the teacher went into great detail for one answer- there must be something ‘special’ about that choice option.
Multiple-Choice Item # 2 - flaw(s)? This question has an unintended clue occur at the very end. The word "an" gives a clue that the answer must be A.
Evidence: It is clear that this question supports Commandment 3. Thou shall not provide students with unintended clues regarding appropriate responses. The students who know there grammar very well will immediately choose “A” – they do not have to know any information about No Child Left Behind to guess correctly!
Multiple-Choice Item # 3 - flaw(s)? The flaw for this question would be using "All of the above".
Evidence: On slide 43 it states when you include “all-of-the above” – a student who knows only two of the alternatives are correct can guess that the other two are correct as well. In addition, choices A and B are very similar, so it may throw off the students or it may show them that A and B are both correct, so option D is the correct choice.
Multiple-Choice Item # 4 - flaw(s)? This would have to go along with trying to keep all the answers around the same length. D. is significantly longer than the other three.
Evidence: Slide 40: Students can figure that if the teacher has given so much attention to constructing an answer options there must be something special about it.
Also, this question supports Commandment 3. Thou shall not provide students with unintended clues regarding appropriate responses. Choice D is considerably longer then all the other choices. Students who do not know the answer may infer that if the teacher went into great detail for one answer- there must be something ‘special’ about that choice option.
5. POST YOUR GROUP CONSENSUS HERE (The group leader for this Matching Item activity will be the member whose birthday is closest to Valentine's Day.):
“Find the flaws . . . use evidence from the ‘5 Commandments and/or the Item Writing Guidelines to support your answers.”
On the line to the left of each measurement concept listed in Column A, please write the letter of the most relevant measurement procedure in Column B.
Matching Item # 1 - flaw(s)? First I would mention that there are no extra responses. It is the same on both sides. Next would be the confusing directions.
Evidence: On slide 53 it states the directions for an assessment should always make explicit the basis on which matches are to be made – AND, the number of times a response can be used. Slide 51 states...a student who knows 80% of the matches to the premises may be able to figure out the remaining matches - guessing and/or process of elimination. Either way – this does not give you accurate data about the student. Also, Commandment 1 is broken here because of the unclear directions. Commandment 1. Thou shall not provide opaque directions to students. Unclear test-taking directions can result in confused test takers. Most teachers design test items for topics they know very well - and often provide only sketchy directions for students.